Migrate to pnpm workspaces with automated npm publishing by dannon · Pull Request #89 · galaxyproject/galaxy-visualizations

dannon · 2025-06-16T15:46:01Z

Summary

pnpm migration: Replaced Yarn + Lerna with pnpm workspaces. All 48 packages resolve and install successfully. Root package.json has convenience scripts (build:all, test:all, lint:all, clean).
Automated publishing: New publish-packages.yml workflow detects version bumps on merge to main and publishes changed packages to npm under @galaxyproject. Manual publish workflow with dry-run support also included.
Test workflow updated: Existing test workflow adapted for pnpm (pnpm store caching, pnpm install --frozen-lockfile from root, Node 22). Kept the dynamic discovery, Playwright separation, and Python dependency support.
Prettier consolidated: Moved 30 identical per-package prettier.config.js files to a single root config. Added .prettierignore.
Package cleanup: Scoped webr as @galaxyproject/webr, replaced hardcoded yarn calls in build scripts, removed 12 stale per-package yarn.lock files.

Build/test verification

pnpm install — all 48 workspace packages resolve
33/40 packages with build scripts build successfully (7 failures are pre-existing dependency issues)
4/4 unit-test-only packages pass locally (Playwright tests require CI server environment)

Notes

Requires NPM_TOKEN secret in repo settings for publishing to work
No package versions were changed (won't trigger any publishes on merge)

Test plan

CI test workflow passes on this PR
Verify pnpm install && pnpm build:all works locally
After merge, confirm publish workflow runs but skips (no version changes)
Test manual publish workflow with dry-run on a single package

davelopez

This is very helpful! Thank you!

.github/workflows/manual-publish.yml

Scope webr as @galaxyproject/webr and remove private:true for consistency with all other packages. Replace hardcoded yarn calls in alignmentviewer and annotateimage build scripts with generic equivalents that work under any package manager.

pnpm workspaces will manage all dependencies from the root lockfile, so these per-package lock files are no longer needed.

Replace lerna.json and yarn.lock with pnpm-workspace.yaml and pnpm-lock.yaml. Root package.json now has pnpm convenience scripts (build:all, test:all, lint:all, clean) and the packageManager field set to pnpm@10.12.1. All 48 workspace packages resolve successfully.

All 30 per-package prettier.config.js files were byte-identical. Moved the shared config to root and added a .prettierignore. Removed the explicit --config flags from 7 package prettier scripts since prettier will find the root config automatically.

Switch from per-package npm install to root-level pnpm install with frozen lockfile. Replace npm/npx commands with pnpm equivalents. Update cache strategy to use pnpm store instead of per-package node_modules. Bump to Node 22.

publish-packages.yml triggers on push to main, detects package.json version changes, and publishes updated packages to npm under the @galaxyproject scope. manual-publish.yml provides a workflow_dispatch for publishing individual packages with dry-run support.

Add repository structure, pnpm development setup commands, publishing workflow documentation, and contributing guidelines.

setup-node@v5 tries to detect and cache the package manager on setup, so pnpm needs to be installed first. Also bumped pnpm/action-setup from v4 to v5 (latest).

dannon · 2026-03-19T02:10:55Z

TODO: Set up npm trusted publishing (OIDC)

The publish workflows currently reference an NPM_TOKEN secret. Before enabling publishing, we should switch to npm trusted publishing via OIDC instead -- no long-lived tokens needed, npm verifies the publish came from an authorized GitHub Actions workflow.

Setup steps:

Use the npm trust CLI to configure trusted publishing in bulk for all @galaxyproject packages:

for pkg in $(ls packages/); do
  npm trust github \
    --package "@galaxyproject/$pkg" \
    --repository "galaxyproject/galaxy-visualizations" \
    --workflow "publish-packages.yml" \
    --yes
  sleep 2
done

First run prompts for 2FA -- check "skip 2FA for 5 minutes" on the npm site to let the rest go through.

Repeat with --workflow "manual-publish.yml" for the manual publish workflow (if npm supports multiple workflows per publisher -- otherwise we may need to consolidate).
Update the workflow YAML files to use OIDC: add permissions: id-token: write, use npm publish --access public --provenance, and drop the NPM_TOKEN env var.

Caveat: Initial version of any never-published package can't use OIDC -- needs a one-time token-based publish first.

Ref: https://docs.npmjs.com/trusted-publishers/ / https://docs.npmjs.com/cli/v11/commands/npm-trust/

davelopez reviewed Jun 16, 2025

View reviewed changes

.github/workflows/manual-publish.yml Outdated Show resolved Hide resolved

dannon changed the title ~~Automatic visualization publishing, package standardization and~~ Automatic visualization publishing, package standardization and improved tooling Jun 16, 2025

dannon added 7 commits March 18, 2026 15:40

Remove per-package yarn.lock files

3e61ffb

pnpm workspaces will manage all dependencies from the root lockfile, so these per-package lock files are no longer needed.

Consolidate prettier config to root

7bf2290

All 30 per-package prettier.config.js files were byte-identical. Moved the shared config to root and added a .prettierignore. Removed the explicit --config flags from 7 package prettier scripts since prettier will find the root config automatically.

Update test workflow for pnpm

22022c1

Switch from per-package npm install to root-level pnpm install with frozen lockfile. Replace npm/npx commands with pnpm equivalents. Update cache strategy to use pnpm store instead of per-package node_modules. Bump to Node 22.

Update README with development setup and publishing docs

70d8047

Add repository structure, pnpm development setup commands, publishing workflow documentation, and contributing guidelines.

dannon force-pushed the publishing branch from b341a3a to 70d8047 Compare March 19, 2026 02:01

Fix trailing comma in heatmap vite config

e387a18

dannon marked this pull request as ready for review March 19, 2026 02:02

dannon changed the title ~~Automatic visualization publishing, package standardization and improved tooling~~ Migrate to pnpm workspaces with automated npm publishing Mar 19, 2026

dannon added 2 commits March 18, 2026 22:02

Gitignore test-results and playwright-report directories

69ffd5c

Fix CI: setup pnpm before Node, bump to action-setup v5

dca2f94

setup-node@v5 tries to detect and cache the package manager on setup, so pnpm needs to be installed first. Also bumped pnpm/action-setup from v4 to v5 (latest).

dannon mentioned this pull request Mar 20, 2026

Add bamjs BAM file viewer #88

Draft

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate to pnpm workspaces with automated npm publishing#89

Migrate to pnpm workspaces with automated npm publishing#89
dannon wants to merge 10 commits intomainfrom
publishing

dannon commented Jun 16, 2025 •

edited

Loading

Uh oh!

davelopez left a comment

Uh oh!

Uh oh!

dannon commented Mar 19, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

dannon commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Build/test verification

Notes

Test plan

Uh oh!

davelopez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dannon commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dannon commented Jun 16, 2025 •

edited

Loading

dannon commented Mar 19, 2026 •

edited

Loading